Nowadays, Video semantic Segmentation is used in many applications such as automatic driving, navigation systems, virtual reality systems, etc. In recent years, significant progress has been observed in semantic Segmentation of images. Since the consecutive frames of a Video must be processed with high speed, low latency, and in real time, using semantic image Segmentation methods on individual Video frames is not efficient. Therefore, semantic Segmentation of Video frames in real time and with appropriate accuracy is a challenging topic. In order to encounter the mentioned challenge, a Video semantic Segmentation framework has been introduced. In this method, the previous frames semantic Segmentation has been used to increase speed and accuracy. For this manner we use the optical flow (change of continuous frames) and a GRU deep neural network called ConvGRU. One of the GRU input is estimation of current frames semantic Segmentation (resulting from a pre-trained convolutional neural network), and the other one is warping of previous frames semantic Segmentation along the optical flow. The proposed method has competitive results on accuracy and speed. This method achieves good performances on two challenging Video semantic Segmentation datasets, particularly 83. 1% mIoU on Cityscapes and 79. 8% mIoU on CamVid dataset. Meanwhile, in the proposed method, the semantic Segmentation speed using a Tesla P4 GPU on the Cityscapes and Camvid datasets has reached 34 and 36. 3 fps, respectively.

Image and Video completion are essential tasks in the field of image and Video processing, often used for restoring damaged regions in images and Video frames. The primary challenge in these tasks is to complete them in such a way that they do not introduce noticeable artifacts or inconsistencies to the viewer. While image completion focuses on filling in missing parts in a static context, Video completion requires additional considerations due to the Temporal dimension. The motion of objects and the preservation of Temporal consistency are critical factors in Video completion. This research proposes a novel method for image and Video completion based on Neutrosophic theory, which handles uncertainty in both spatial and intensity domains. Neutrosophy is utilized to interpret the indeterminacy present in images, allowing for more accurate Segmentation and better handling of incomplete data. The proposed method first segments the image using Neutrosophic-based Segmentation and then uses the segmented information to guide the completion of missing regions. For Video completion, a two-step approach is introduced that separates static backgrounds from moving objects. The background is reconstructed using image completion based on Neutrosophic-based Segmentation, and the foreground is completed by identifying appropriate data that best match the missing parts; this data is chosen using a contour-based method, which this method applies neutrosophic sets to get to the most suitable data. The novelty of the approach lies in several key contributions: 1) the use of Neutrosophic theory to handle spatial and intensity uncertainties, 2) a Neutrosophic-based similarity measure for image Segmentation, 3) a new metric for finding the most suitable patch for hole-filling, and 4) a novel method for preserving boundaries and uniformity in Video completion, particularly in the presence of moving objects. Experimental results demonstrate the effectiveness of the proposed methods, with improved visual quality and reduced inconsistencies compared to previous state-of-the-art methods. However, challenges remain in applying the method to highly detailed images with many classes and handling dynamic backgrounds.

Recognition of visual events as a Video analysis task has become popular in machine learning community. While the traditional approaches for detection of Video events have been used for a long time, the recently evolved deep learning based methods have revolutionized this area. They have enabled event recognition systems to achieve detection rates which were not reachable by traditional approaches. Convolutional neural networks (CNNs) are among the most popular types of deep networks utilized in both imaga and Video recognition tasks. They are initially made up of several convolutional layers, each of which followed by proper activation and possibly pooling layers. They often encompass one or more fully connected layers as the last layers. The favorite property of them in this work is the ability of CNNs to extract mid-level features from Video frames. Actually, despite traditional approaches based on low-level visual features, the CNNs make it possible to extract higher level semantic features from the Video frames. The focus of this paper is on recognition of visual events in Video using CNNs. In this work, image trained descriptor s are used to make Video recognition can be done with low computational complexity. A tuned CNN is used as the frame descriptor and its fully connected layers are utilized as concept detectors. So, the featue maps of activation layers following fully connected layers act as feature vectors. These feature vectors (concept vectors) are actually the mid-level features which are a better Video representation than the low level features. The obtained mid-level features can partially fill the semantic gap between low level features and high level semantics of Video. The obtained descriptors from the CNNs for each Video are varying length stack of feature vectors. To make the obtained descriptors organized and prepared for clasification, they must be properly encoded. The coded descriptors are then normalized and classified. The normaliztion may consist of conventional and normalization or more advanced power-law normalization. The main purpose of normalization is to change the distribution of descriptor values in a way to make them more uniformly distributed. So, very large or very small descriptors could have a more balanced impact on recognition of events. The main novelty of this paper is that spatial and Temporal information in mid-level features are employed to construct a suitable coding procedure. We use Temporal information in coding of Video descriptors. Such information is often ignored, resulting in reduced coding efficiency. Hence, a new coding is proposed which improves the trade-off between the computation complexity of the recognition scheme and the accuracy in identifying Video events. It is also shown that the proposed coding is in the form of an optimization problem that can be solved with existing algorithms. The optimization problem is initially non-convex and not solvable with the existing methods in polynomial time. So, it is transformed to a convex form which makes it a well defined optimization problem. While there are many methods to handle these types of convex optimization problems, we chose to use a strong convex optimization library to efficiently solve the problem and obtain the Video descriptors. To confirm the effectiveness of the proposed descriptor coding method, extensive experiments are done on two large public datasets: Columbia consumer Video (CCV) dataset and ActivityNet dataset. Both CCV and ActivityNet are popular publically available Video event recognition datasets, with standard train/test splits, which are large enough to be used as reasonable benchmarks in Video recognition tasks. Compared to the best practices available in the field of detecting visual events, the proposed method provides a better model of Video and a much better mean average precision, mean average recall, and F score on the test set of CCV and ActivityNet datasets. The presented method not only improves the performance in terms of accuracy, but also reduces the computational cost with respect to those of the state of the art. The experiments vividly confirm the potential of the proposed method in improving the performance of visual recognition systems, especially in supervised Video event detection.

Video magnification is a computational procedure to reveal subtle variations during Video frames that are invisible to the naked eye. A new spatio-Temporal method which makes use of connectivity based mapping of the wavelet sub-bands is introduced here for exaggerating of small motions during Video frames. In this method, firstly the wavelet transformed frames are mapped to connectivity space and then decomposed into different spatial frequency bands by applying Laplacian Pyramid to determine the pixels having more chance to be a part of a movement. Finally each candidate is partially magnified based on its time history. The performance of the proposed method is evaluated on real Videos which contain several subtle motions. Parameters for performance evaluation are presented and obtained results are compared with one of the state-of-the-art Video magnification methods. Increased true positive rate parallel with simultaneous decrease in false positive rate confirms the effectiveness of the proposed method in amplifying subtle motions.

Despite error resilient methods that are applied on Video data in transmitter side, occurring error along Video data transferring for communication channels is inevitable. Error concealment is a useful method for improving the quality of damaged Videos on the receiver side. In this paper, a fast and hybrid boundary matching algorithm is presented for more accurate estimating of damaged motion vectors (MVs) from received Video. According to the preference list of error concealment, the proposed algorithm performs the error concealment for each damaged macroblock (MB). In the presented method, the boundary distortion is calculated for each pixel from each candidate MB's boundary with use of proposed hybrid boundary matching criterion. Then, depending on the accuracy of each adjacent boundary from damaged MB, a special weight is given to them through match process. Finally, the list of error concealment preference is updated and the candidate MV with the lowest boundary distortion is selected as the MV of damaged MB. Experimental results show that the proposed algorithm increases the average of PSNR for different test sequences more than 1.8 dB in comparison with reference methods and without significant increasing in calculation time and with improving the quality of reconstructed Videos.